{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 知识准备\n",
    "\n",
    "* NumPy\n",
    "* k-近邻算法原理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import operator\n",
    "\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# k-近邻算法原理回顾\n",
    "\n",
    "**找k个距离x最近的样本，以此预测x的类别。**\n",
    "\n",
    "1. 确定超参k\n",
    "    * 交叉验证 D->m份  平均错误率  标准差\n",
    "2. 确定距离度量\n",
    "    * $L_p = \\left[\\displaystyle\\sum_{k=1}^{d}|x_{ik} - x_{jk}|^p\\right]^{\\frac{1}{p}}$ \n",
    "    * $p = 1$ 曼哈顿距离 $L_1 = \\displaystyle\\sum_{k=1}^{d}|x_{ik} - x_{jk}|$\n",
    "    * $p = 2$ 欧氏距离 $L_2 = \\left[\\displaystyle\\sum_{k=1}^{d}|x_{ik} - x_{jk}|^2\\right]^{\\frac{1}{2}}$\n",
    "    * $p = \\infty$ 切氏距离 $L_\\infty = \\mathop{argmax}\\limits_{1\\leq k \\leq d}|x_{ik} - x_{jk}|$\n",
    "3. 数据预处理 - 数据标准化\n",
    "    * z-score规范化 $x'_i = \\frac{x_i - \\mu}{\\sigma}$\n",
    "    * 最大-最小规范化 $x'_i = \\frac{x_i - min}{max - min}$\n",
    "4. 计算距离，找到x的k个近邻样本\n",
    "5. 预测样本类别\n",
    "    * 多数选举 最近邻\n",
    "    * 加权重\n",
    "\n",
    "\n",
    "# 案例简介\n",
    "## 背景\n",
    "海伦一直使用在线约会网站寻找适合自己的约会对象，她发现曾交往过三种类型的人：\n",
    "* 不喜欢的人\n",
    "* 魅力一般的人\n",
    "* 极具魅力的人\n",
    "\n",
    "她觉得可以在周一到周五约会那些魅力一般的人，而周末则更喜欢与那些极具魅力的人为伴。\n",
    "\n",
    "海伦希望分类软件可以更好地帮助她将匹配对象划分到确切的分类中。\n",
    "## 数据描述\n",
    "海伦收集的1000条约会数据，包含三个特征分别为\n",
    "* 每年获得的飞行常客里程数\n",
    "* 玩视频游戏所消耗时间百分比\n",
    "* 每周消费的冰淇淋公升数\n",
    "\n",
    "标签为\n",
    "* 不喜欢\n",
    "* 有些喜欢\n",
    "* 非常喜欢\n",
    "\n",
    "# 实现步骤\n",
    "\n",
    "1. 准备训练集D\n",
    "\n",
    "2. 对训练集D的属性进⾏预处理 **最大-最小规范化**\n",
    "\n",
    "3. 训练集D内找到预处理的样本x的前k个近邻 **欧氏距离**\n",
    "\n",
    "4. 结合**多数选举**的分类规则，对x的类别y进⾏预测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 准备训练集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def file2matrix(filename):\n",
    "    \"\"\"\n",
    "    函数说明：打开解析文件，对数据进行分类，\n",
    "        1代表不喜欢，2代表魅力一般，3代表极具魅力\n",
    "\n",
    "    Parameters:\n",
    "        filename - 文件名\n",
    "\n",
    "    Returns:\n",
    "        returnMat - 特征矩阵\n",
    "        classLabelVector - 分类label向量\n",
    "    \"\"\"\n",
    "    # 打开文件\n",
    "    with open(filename, 'r') as fr:\n",
    "        # 读取文件所有内容\n",
    "        arrayOflines = fr.readlines()\n",
    "   \n",
    "    # 得到文件行数\n",
    "    numberOfLines = len(arrayOflines)\n",
    "    # 返回的NumPy矩阵numberOfLines行，3列\n",
    "    returnMat = np.zeros((numberOfLines, 3))\n",
    "    # 创建分类标签向量\n",
    "    classLabelVector = []\n",
    "    # 行的索引值\n",
    "    index = 0\n",
    "    # 读取每一行\n",
    "    for line in arrayOflines:\n",
    "        # 去掉每一行首尾的空白符，例如'\\n','\\r','\\t',' '\n",
    "        line = line.strip()\n",
    "        # 将每一行内容根据'\\t'符进行切片,本例中一共有4列\n",
    "        listFromLine = line.split('\\t')\n",
    "        # 加入字符串转float的代码\n",
    "        # 将数据的前3列进行提取保存在returnMat特征矩阵中\n",
    "        # returnMat[index,:] = listFromLine[0:3]\n",
    "        returnMat[index,:] = [float(i) for i in listFromLine[:-1]]\n",
    "        # 根据文本内容进行分类1：不喜欢；2：一般；3：喜欢\n",
    "        if listFromLine[-1] == 'didntLike':\n",
    "            classLabelVector.append(1)\n",
    "        elif listFromLine[-1] == 'smallDoses':\n",
    "            classLabelVector.append(2)\n",
    "        elif listFromLine[-1] == 'largeDoses':\n",
    "            classLabelVector.append(3)\n",
    "        index += 1\n",
    "    # 返回标签列向量以及特征矩阵\n",
    "    return returnMat, np.array(classLabelVector)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 对训练集D的属性进行预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def autoNorm(dataSet):\n",
    "    \"\"\"\n",
    "    函数说明：对数据进行归一化(或称为，规范化)。\n",
    "    使用最小最大规范化，将数据值映射到[0,1]之间。\n",
    "    规范化后的x = (x-min)/(max-min)\n",
    "\n",
    "    Parameters:\n",
    "        dataSet - 特征矩阵\n",
    "\n",
    "    Returns:\n",
    "        normDataSet - 归一化后的特征矩阵\n",
    "        ranges - 数据范围\n",
    "        minVals - 数据最小值\n",
    "    \"\"\"\n",
    "    # 获取矩阵每一列的最小值\n",
    "    minVals = dataSet.min(0)\n",
    "    # 获取矩阵每一列的最大值\n",
    "    maxVals = dataSet.max(0)\n",
    "    # 最大值和最小值的范围\n",
    "    ranges = maxVals - minVals\n",
    "    # shape(dataSet)返回dataSet的矩阵行列数\n",
    "    normDataSet = np.zeros(np.shape(dataSet))\n",
    "    # numpy函数shape[0]返回dataSet的行数\n",
    "    m = dataSet.shape[0]\n",
    "    # 原始值减去最小值（x-xmin）  \n",
    "    # np.tile()将minVals复制成m行\n",
    "    normDataSet = dataSet - np.tile(minVals, (m, 1))\n",
    "    # 差值除以最大值和最小值的差值（x-xmin）/（xmax-xmin）\n",
    "    normDataSet = normDataSet / np.tile(ranges, (m, 1))\n",
    "    # 归一化数据结果，数据范围，最小值\n",
    "    return normDataSet, ranges, minVals"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 寻找样本x的前k个近邻"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def classify0(inX, dataSet, labels, k):\n",
    "    \"\"\"\n",
    "    函数说明：kNN算法，分类器\n",
    "\n",
    "    Parameters:\n",
    "        inX - 用于分类的数据（测试集）\n",
    "        dataSet - 用于训练的数据（训练集）（n*3矩阵）\n",
    "        labels - 分类标准（n*1维列向量）\n",
    "        k - kNN算法参数，选择距离最小的k个点\n",
    "\n",
    "    Returns:\n",
    "        sortedClassCount[0][0] - 分类结果\n",
    "    \"\"\"\n",
    "    # 返回dataSet的样本数量\n",
    "    dataSetSize = dataSet.shape[0]\n",
    "    # 将inX重复dataSetSize次并排成一列\n",
    "    xTileMat = np.tile(inX, (dataSetSize, 1)) \n",
    "    \n",
    "    # 特征相减后，对每个元素取平方\n",
    "    sqDiffMat = (xTileMat- dataSet)**2\n",
    "    # sum()所有元素相加，sum(0)列相加，sum(1)行相加\n",
    "    sqDistances = sqDiffMat.sum(axis=1)\n",
    "    # 开方，计算出距离\n",
    "    distances = sqDistances**0.5\n",
    "    \n",
    "    # argsort函数返回的是distances值从小到大的--索引值\n",
    "    sortedDistIndicies = distances.argsort()\n",
    "    # 定义一个记录类别次数的字典\n",
    "    classCount = {}\n",
    "    # 选择距离最小的k个点\n",
    "    for i in range(k):\n",
    "        # 取出前k个元素的类别\n",
    "        voteIlabel = labels[sortedDistIndicies[i]]\n",
    "        # 字典的get()方法，返回指定键的值，\n",
    "        # 如果值不在字典中返回0。\n",
    "        # 计算类别次数\n",
    "        classCount[voteIlabel] = classCount.get(voteIlabel, 0) + 1\n",
    "    # reverse降序排序字典\n",
    "    sortedClassCount = sorted(classCount.items(),\n",
    "                              key = operator.itemgetter(1), \n",
    "                              reverse = True)\n",
    "    # 返回次数最多的类别，即所要分类的类别\n",
    "    return sortedClassCount[0][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 结合多数选举规则，预测x的类别y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def datingClassTest():\n",
    "    \"\"\"\n",
    "    函数说明：分类器测试函数\n",
    "\n",
    "    Parameters:\n",
    "        None\n",
    "\n",
    "    Returns:\n",
    "        None\n",
    "    \"\"\"\n",
    "    # 1.准备数据-从文本文件中解析数据、归一化\n",
    "    # 打开文件名\n",
    "    filename = \"datingTestSet.txt\"\n",
    "    # 将返回的特征矩阵和分类向量分别存储到\n",
    "    # datingDataMat和datingLabels中\n",
    "    datingDataMat, datingLabels = file2matrix(filename)\n",
    "    # 数据规范化(最小最大规范化)，返回规范化数据结果，数据范围，最小值\n",
    "    normMat, ranges, minVals = autoNorm(datingDataMat)\n",
    "    # 2.数据集划分\n",
    "    # 获取normMat的行数\n",
    "    m = normMat.shape[0]\n",
    "    # np.random.permutation获得打乱的索引值\n",
    "    idxs = np.random.permutation(m)\n",
    "    # 利用打乱的索引打乱数据\n",
    "    normMat = normMat[idxs]\n",
    "    datingLabels = datingLabels[idxs]\n",
    "    # 取所有数据的10%用于测试\n",
    "    hoRatio = 0.10\n",
    "    # 10%的测试数据的个数\n",
    "    numTestVecs = int(m * hoRatio)\n",
    "    trainX = normMat[numTestVecs:m,:]\n",
    "    trainY = datingLabels[numTestVecs:m]\n",
    "    testX = normMat[:numTestVecs,:]\n",
    "    testY = datingLabels[:numTestVecs]\n",
    "    # 3.在测试集上计算错误率\n",
    "    # 分类错误计数\n",
    "    errorCount = 0\n",
    "    for x,y in zip(testX,testY):\n",
    "        # 前numTestVecs个数据作为测试集，\n",
    "        # 后m-numTestVecs个数据作为训练集\n",
    "        # k选择label数+1（结果比较好）\n",
    "        classifierResult = classify0(\n",
    "            x, trainX, trainY, 4)\n",
    "        print(\"分类结果:%d\\t真实类别:%d\" % (\n",
    "            classifierResult, y))\n",
    "        if classifierResult != y:\n",
    "            errorCount += 1\n",
    "    print(\"错误率:%.2f%%\" % (errorCount/numTestVecs*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 测试数据并评价模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "datingClassTest()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 在线输入样本并预测分类"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def classifyPerson():\n",
    "    \"\"\"\n",
    "    函数说明：通过输入一个人的三个特征，进行分类输出\n",
    "\n",
    "    Parameters:\n",
    "        None\n",
    "\n",
    "    Returns:\n",
    "        None\n",
    "    \"\"\"\n",
    "    # 输出结果\n",
    "    resultList = ['讨厌', '有些喜欢', '非常喜欢']\n",
    "    # 三维特征用户输入\n",
    "    percentTats = float(input(\"玩视频游戏所消耗时间百分比：\"))\n",
    "    ffMiles = float(input(\"每年获得的飞行常客里程数：\"))\n",
    "    iceCream = float(input(\"每周消费的冰淇淋公升数：\"))\n",
    "    # 打开的文件名\n",
    "    filename = \"datingTestSet.txt\"\n",
    "    # 打开并处理数据\n",
    "    datingDataMat, datingLabels = file2matrix(filename)\n",
    "    # 训练集归一化\n",
    "    normMat, ranges, minVals = autoNorm(datingDataMat)\n",
    "    # 生成NumPy数组，测试集\n",
    "    inArr = np.array([percentTats, ffMiles, iceCream])\n",
    "    # 测试集归一化\n",
    "    norminArr = (inArr - minVals) / ranges\n",
    "    # 返回分类结果\n",
    "    classifierResult = classify0(norminArr, \n",
    "                                 normMat, datingLabels, 4)\n",
    "    # 打印结果\n",
    "    print(\"你可能%s这个人\" % (\n",
    "        resultList[classifierResult - 1])) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "玩视频游戏所消耗时间百分比：1\n",
      "每年获得的飞行常客里程数：1\n",
      "每周消费的冰淇淋公升数：1\n",
      "你可能有些喜欢这个人\n"
     ]
    }
   ],
   "source": [
    "classifyPerson()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 交叉验证 寻找最佳的k"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"准备好打乱的数据集\n",
    "return:\n",
    "    norm_mat特征矩阵\n",
    "    labels 标签数组\n",
    "\"\"\"\n",
    "def pre_data():\n",
    "    data_mat, labels = file2matrix('datingTestSet.txt')\n",
    "    norm_mat, min_vector, max_vector = autoNorm(data_mat)\n",
    "    n = norm_mat.shape[0]\n",
    "    idxs = np.random.permutation(n)\n",
    "    norm_mat = norm_mat[idxs]\n",
    "    labels = labels[idxs]\n",
    "    return norm_mat, labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"交叉验证\n",
    "\n",
    "return：\n",
    "    平均错误率\n",
    "\"\"\"\n",
    "def cross_validation(norm_mat, labels, k):\n",
    "    # 交叉验证，如把样本分成10份\n",
    "    n = norm_mat.shape[0]\n",
    "    test_n = int(n * 0.1)\n",
    "    error_list = []\n",
    "    for i in range(10):\n",
    "        #print(\"i=\", i)\n",
    "        test_start = test_n * i\n",
    "        test_end = test_n * (i + 1)\n",
    "        # 测试集\n",
    "        testx = norm_mat[test_start:test_end,:]\n",
    "        testy = labels[test_start:test_end]\n",
    "        # 训练集\n",
    "        #trainx\n",
    "        trainx1 = norm_mat[:test_start,:]\n",
    "        trainx2 =  norm_mat[test_end:,:]\n",
    "        if(trainx1.shape[0] == 0):\n",
    "            trainx = trainx2\n",
    "        elif(trainx2.shape[0] == 0):\n",
    "            trainx = trainx1\n",
    "        else:\n",
    "            trainx = np.r_[trainx1, trainx2]\n",
    "        # trainy\n",
    "        trainy1 = labels[:test_start]\n",
    "        trainy2 = labels[test_end:]\n",
    "        if(trainy1.shape[0] == 0):\n",
    "            trainy = trainy2\n",
    "        elif(trainy2.shape[0] == 0):\n",
    "            trainy = trainy1\n",
    "        else:\n",
    "            trainy = np.r_[trainy1, trainy2]\n",
    "        # 计算一次错误率\n",
    "        error_count = 0\n",
    "        for x,y in zip(testx,testy):\n",
    "            xlabel = classify0(x, trainx, trainy, k)\n",
    "            if xlabel != y:\n",
    "                error_count += 1\n",
    "        error1 = error_count/test_n*100\n",
    "        # 将本次错误率保存到error_list\n",
    "        error_list.append(error1)\n",
    "    # 算平均错误率\n",
    "    err_u = np.array(error_list).sum(0) / len(error_list)\n",
    "    return err_u"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "norm_mat, labels = pre_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.6"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cross_validation(norm_mat, labels, k=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEGCAYAAAB1iW6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1d3H8c8hASSgsgZZkiDWohAVLAUR61q1ooJLtQhWa7WRxwVUqg801tYqLa111weL2roQRcUFanGra11rUJSwqKgQNg2bbGEL/J4/zgwkISFDMjN35ub7fr3ySubOzZ3fZfnm5MxZnJkhIiLpr0nQBYiISHwo0EVEQkKBLiISEgp0EZGQUKCLiIREZlAv3L59e+vWrVtQLy8ikpZmzJixwsw61PRcYIHerVs3iouLg3p5EZG05JxbWNtz6nIREQkJBbqISEgo0EVEQkKBLiISEgp0EZGQUKCLiCRJURF06wZNmvjPRUXxvX5gwxZFRBqToiIoKIDycv944UL/GGD48Pi8hlroIiJJUFjow/wGbuRwZgD+cWFh/F5DLXQRkSQoLYXBTOVGfk8mFXzED3Ycjxe10EVEkqB3l+VMpICP6c1N/HbH8dzc+L2GWugiIolmxnOdRtB68XecwKtspRkAWVkwblz8XkYtdBGRRCsqIvfDZ5gz9CbW5+XjHOTlwcSJ8XtDFNRCFxFJrMWL4YorYOBA+kwazYKMxL2UWugiIoliBr/8JWzdCg89BBkJTHNiDHTnXGvn3BTn3Dzn3Fzn3IBqzx/rnFvjnJsZ+bghMeWKiKSRCRPglVfg1lvhe99L+MvF2uVyJ/Cimf3UOdcMyKrhnP+Y2WnxK01EJI3Nnw/XXgsnnwyXXpqUl6wz0J1z+wBHA78AMLMtwJbEliUiksa2bYMLL4RmzeDBB8G5pLxsLF0u3YHlwD+ccx875x5wzrWs4bwBzrlPnHMvOOd61XQh51yBc67YOVe8fPnyhtQtIpK6/vpXePdduOce6NIlaS8bS6BnAocDE8ysD7ABGFPtnI+APDM7DLgbeK6mC5nZRDPra2Z9O3SocUs8EZH09umn8Nvfwtlnw7BhSX3pWAJ9MbDYzD6IPJ6CD/gdzGytma2PfD0daOqcax/XSkVEUt2WLXDBBdCmjX9DNEldLVF1BrqZfQMscs71iBw6AZhT+Rzn3H7O+cqdc/0i110Z51pFRFLbjTfCJ5/A/fdDAL0QsY5yuRIoioxw+Qq4yDk3AsDM7gN+CvyPc64C2AgMNTNLRMEiIinp/fdh/Hi46CIYPDiQElxQudu3b18rLi4O5LVFROKqvBz69IFNm2DWLNhnn4S9lHNuhpn1rek5Tf0XEWmoMWPg88/htdcSGuZ10dR/EZGGePVVuPtuGDkSjjsu0FIU6CIi9bVmje8z79ED/vSnoKtRl4uISL2NGgVLl/pJRFk1rYiSXGqhi4jUx9Sp8PDDMHYs9OsXdDWAAl1EZM8tXw4FBX5ky29/W/f5SaIuFxGRPWHmV0/87jv/hmizZkFXtIMCXURkT0yaBM8+C3/+M+TnB11NFepyERGJ1aJFcOWVMHAgjB4ddDW7UKCLiMTCDC6+GCoq/JuhCd5Orj7U5SIiEovodnITJsABBwRdTY3UQhcRqcsXXyR9O7n6UKCLiOxOQNvJ1Ye6XEREdueWW+C996CoKKnbydWHWugiIrX59FO44Qb46U/hvPOCrqZOCnQRkZpEt5Nr2zaQ7eTqQ10uIiI1iW4nN20atE+PLZLVQhcRqa7ydnKnnx50NTFToIuIVLZhg+9q6doV7rgj6Gr2iLpcREQqGzPGjzsPeDu5+lALXUQk6tVX4Z57/MYVAW8nVx8KdBERSLnt5OpDXS4iIlB1O7kWLYKupl7UQhcRee65lNtOrj4U6CLSuJWVpeR2cvWhLhcRabzMYMQI33/+2msptZ1cfSjQRaTxim4n95e/pNx2cvWRVl0uRUXQrRs0aeJn4rZv77/u1s0/JyJSl2iO5LpFrL3wSsq+fxRcc03QZcVF2gR6UZHv5lq40P+WtHKl/zDzxwoKFOoisns7c8R4gItpYhUcX/oQRZNTbzu5+nBmFsgL9+3b14qLi2M+v1s3H9x9+IiLeXDH8XXszR+4gY1kkZcHCxbEv1YRCYdojlzF7dzONYxgAn9jRFplh3Nuhpn1rem5tOlDLy31n3NYxLk8CUBTttKaNbzEybzBcTvOERGpSelC4zf8kXFcz1QG8zf8dnJhyY606XLJzfWfpzGEbJaTzXJO5BUAWrG+yjkiIrvYupWiVgWM43oe4eecw1OAX+M8LNmRNoE+bhxkZVU9toGWALRkA1lZ/hwRkV2sWweDB3Pe+gcYn3k9F/IwW/FDFMOUHWkT6MOHw8SJkJfnNw5p1w6atfaBntt2AxMn+nNERKpYtgyOOQZeeQUmTiTnoZvIy3M45/MkTNmRNm+K1mjFCujQAe68E0aOjE9hIhIec+bAKaf4IXFPPeW/TnO7e1M0bVroNWrpW+hs2BBsHSKSet54A4480u8N+tZboQjzusQU6M651s65Kc65ec65uc65AdWed865u5xz851znzrnDk9MudXstZfvf1Ggi0hljz0GJ58MnTv77eQOT04kBS3WFvqdwItmdhBwGDC32vOnAAdGPgqACXGrcHec8610BbqIgJ9pOH687xQfMADeecd3lDcSdQa6c24f4Gjws3nMbIuZfVfttCHAI+a9D7R2znWKe7U1UaCLCEBFBVx2mV8C97zz4KWXoE2boKtKqlha6N2B5cA/nHMfO+cecM61rHZOF2BRpceLI8eqcM4VOOeKnXPFy5cvr3fRVSjQRWT9ejjjDLjvPr8n6KRJ0Lx50FUlXSyBngkcDkwwsz7ABmBMtXNcDd+3y/AZM5toZn3NrG+HDh32uNgatWwJ5eXxuZaIpJ9vvoFjj4UXXoAJE/z2cU3Se7xHfcVy14uBxWb2QeTxFHzAVz8np9LjrsDShpcXA7XQRRqvefN8X/ncuTB1ql/bvBGrM9DN7BtgkXOuR+TQCcCcaqdNAy6IjHY5AlhjZsviW2otFOgijdN//uOHJZaXw5tvwmmnBV1R4GJdnOtKoMg51wz4CrjIOTcCwMzuA6YDg4D5QDlwUQJqrVnLln6CkYg0Hk88ARdcAPvv77ta9t8/6IpSQkyBbmYzgeozk+6r9LwBl8exrtiphS7SeJjBrbfCtdfCUUf5bpa2bYOuKmWk/zsHCnSRxmHbNrjySh/m557r12ZRmFehQBeR1FdeDmedBffe6wP98cf9THGpIm02uKhVNNDN/MxREQmXsjI4/XQoLoZ77oHLg+ndTQfhCPRt2/wCPI1wIoFIqH3+uV9Ua9kyeOYZGDIk6IpSWjgCHXwrXYEuEh7vvAODB0NGBrz+OvTvH3RFKS8cfeigfnSRMHn6aTjhBL+TzXvvKcxjpEAXkdRyxx1wzjnwgx/Au+/CAQcEXVHaSP9Aj240qkAXSW/btsFVV8HVV/sRLf/+N7RvH3RVaSX9A10tdJH0t3GjH1t+550+0J98Elq0CLqqtKNAF5GkKyqCbt38ooh9claw/LAT4NlnfXfLbbc12tUSGypco1xEJOUVFUFBgZ8rdADzeXLxKbRiMW+NmsLRo84Kury0lv4/BhXoImmlsNCHeT8+4D0G0IbVHM9rXPCcwryhFOgiklSlpTCE53id41jDvgzgPd5nAKWlQVeW/hToIpJUv219N89wFp9yKAN4j/kcCEBubsCFhUD6B7qGLYqkh+3b4de/5sbVI3k+YwjH8xor8FtRZmXBuHEB1xcC6R/oGRl+1TUFukjq2rQJhg71a5lfeSXr/zGF7LwsnIO8PJg4EYYPD7rI9Jf+o1xAG0WLpLKVK/2iWu+84wP96qsZ5hzDfh50YeETnkBXC10k9Xz1FQwaBAsW+MlC55wTdEWhpkAXkcT48EO/cXNFhZ/Gf9RRQVcUeunfhw4KdJFU889/wrHH+v+b776rME8SBbqIxNeECXDGGdCrl1/6tkePoCtqNBToIhIf27fD//4vXHYZnHqq35SiY8egq2pU1IcuIg23eTP84hcwebIP9Lvu8kOKJakU6CLSMKtX+y6Wt96Cv/wFfv1rbdgeEAW6iNTfggV+WOKXX8Ljj/vJQxIYBbqI1M+MGX5Y4qZN8MorcPTRQVfU6IXnTdHNm/0WViKSeNOnwzHHQPPmfliiwjwlhCfQQa10SUuVd+/p1s0/TmkTJ8LgwX444vvvw8EHB12RRCjQRQIU3b1n4UIw858LClI01M387hSXXgonnwxvvgn77Rd0VVKJAl0kQIWF0KJ8BTdTyPXcRB8+orzcKCwMurJqtmyBn/8c/vhH+NWvYOpUaNUq6KqkmnC8Kao10SUdVVQweOEEbuQG9mEtGWznJm5gKZ14YeEgePZU+PGPYe+9g63zu+/grLP8RKFx42DsWA1LTFFqoYsE4dVXoXdv7mIkH/JDDmEWHfmGC3mItzmKc9xTPkTbtfOhfvvt8Pnnya+ztNSvw/L22/Doo/Cb3yjMU5gCXSSZFiyAs8/2IV1ezptXP8eZLV5iLj0poyOPcCEXZT3J8/9Y4VvEo0bB0qVwzTX+TcgDD4SrroKXX/YjuxJp5kw44ghYvBheegnOPz+xrycNpkAXSYbycvjd7/yIkBdf9F0Xc+ZwzG1DmHi/Iy+PKrv3DLuwqV+t8JZbYM4cv674Pff4QP/b3/ybku3a+Rma998PS5bEt96XXoIf/QgyM33r/Ljj4nt9SQhnZoG8cN++fa24uDg+F/vsMzjoIJg0SftYSWoxg6ee8tPhFy2C887z0+O7dq3/NcvL4bXX/Fjwf/3Ld4sA9O7tF8UaNAj696//WioPPuhHshxyiL9+5871r1Xizjk3w8z61vScWugiifLpp75l+7Of+db0W2/BY481LMzBDwI47TT4v//zXTizZsH48f7N0/HjYeBAv8rh+ef76firVsV2XTO44Qa45BLfJfTWWwrzNBNToDvnFjjnZjnnZjrndmlWO+eOdc6tiTw/0zl3Q/xL3Y2QBHrlCSbt2/uPtJlsEgdpN8GmNitXwuWXQ58+UFIC990HxcW+CyPenIP8fL9s7VtvwfLlfsXDQYN8t8mwYdChg39j809/8j9kKv1WHv0zb+628PTev4CbboJf/tJvUBH06BrZc2ZW5wewAGi/m+ePBZ6P5VrRjx/84AcWN5s3m4HZzTfH75pJNmmSWVaWv42aPrKy/DlhVdP9p909b91qdu+9Zm3bmmVkmF15pdmqVcHVU1Fh9t57Ztdfb3b44Tv/YHNyzC691F6/Zpq1b7He9uE7e5kfm4Hd2PQPNunR7cHVLHUCiq22LK7tiSonpXqgm5llZpqNHRvfayZRXp7/2/gD19tTnG1PcbY9yU/tRF7a8f8wLy/oKhMnev99mGETucSG86i1Y3n63PMbb5gdeqi/ieOPN5s1K+iKdrVkidkDD5ideaZZq1ZmYBtpbkvoZFvItAt4KPT/zsIgHoH+NfARMAMoqOH5Y4GVwCfAC0CvWq5TABQDxbm5ufG9y333NRs5Mr7XTCLnzPai3AxsGR1tFr1sGR2tnL2sH+8b+HPCyjmzXBbYMjraNpwZ2DacvcsRZjfdZPbRR2bbU7DluHCh2bnn7vyJO2VKatZZ3aZNdgL/tlu52v7N8XYCr+xoOIT531kYxCPQO0c+Z0dC++hqz+8DtIp8PQj4oq5rxr2F3rmz2cUXx/eaSZSX5wPNwH7JAwZm7Smz+XS3b8i2PL4OdcspP+c7m0UvW82+djCzrS//td/xO5vZrO/OroLOnc0uucTsmWfM1q4NtuDycrMbbzRr0cJ/3HijP5ZGor8VVf8I87+zMNhdoMf0pqiZLY18LgOeBfpVe36tma2PfD0daOqcax/LteMmzddEHzcOcpuXAVBGNgAr6MCp/ItmbGG6O5W/FK4JssTE2bqVl1qfSw8+4yyeYS49KeaH3JL1e0r+/iEsWwZ//zsceSQ8+eTOGZQnngh33AFffJG8Ws3g6af9ePLf/Q5OPx3mzfOjQ1q0SF4dcTBu3M5VM6KysvxxSVO1JX30A2gJ7F3p63eBn1Q7Zz92jmnvB5RGH9f2EfcWeu/eZqefHt9rJtlro583A+vP+9aunVm7dv7X36HZr9q2jEyzE08027Il6DLja/t2s0svNQN791cPWl6ev+e8vFreEN282ey118xGjzY76KCdzcoDDzQbNcrs5ZfNNm1KTK2zZvn+cTA75BCz119PzOsk0aRJVvefuaQUGtLlAnTHd7N8AswGCiPHRwAjIl9fEXnuE+B94Mi6rhv3QB840P9nS2d//7v/K/nqq9qfKyhIjz7aWP31r/6+xoyp3/d/+aXZ3Xeb/eQnZs2b+2u1amV2xhlm99/v3whsqFWr/IiVjAyzNm38SJatWxt+XZF6aFCgJ+oj7oF+0klm/fvH95rJNn68/ytZv77m58eO9c/fckty60qUZ5/1TcNzzjHbtq3h11u/3mzaNLMRI/zQvGjrvXdvs8JCs3ff9UP5YlVRYfa3v/lflZo0MbvsMrMVKxpep0gD7C7QwzFTFNK+Dx3wk0KysnZOlKru5pvhnHPguuvg2WeTW1u8FRf7SS/9+sHDD/vZRA3VsqXv054wwe8U8emnfjJNq1b+85FH+g0Zfv7zXWZQVp/U9PINb8MPf+inwPfqBR99BPfe6/vuRVJUONZDh3AEelkZZGfX/nyTJj78Fi3ya9a8+aYPnXRTWuqDt2NHv1FCIt5MdM6vRXLIITBmDKxe7WdO/utf8MILft2fJk3gyCP5uPOp3DntVBZuyqcLS/jjwus46abH2dAuh5ZPPOF/iGrJWEkDaqGnkroCHXz4TZ3qw3Dw4J0LM6WLtWv9OiTl5T5cO3ZMzuu2aQNDh/o1vb/91m9s/JvfwIYN9HlyLP/ddCgLyeMzenAWz3AjN/CDrHlw7rkKc0kbCvRUUlbm192oS3a2D8ONG/3qemvXJr62eKio8AtVzZkDU6ZAz57B1JGRAQMG+HVLPvqILizhEu6nmL48zdkczFx+z418vjir7muJpJBwBXp5eZWFh9JOLC30qJ49fSjOm+dDsqIisbU1lBmMHOnXAp8wwY8hTxFN8zrzIJdwNs9wIY+wgP0ByM0NuDCRPRSuQDfzrdZ0ZLZngQ5+idMJE3xIjhyZ2j/M7rjD13rttX6T4RSiCTYSFuEKdEjfbpc1a2Dr1j0LdPBrV193nQ/LO+5ITG0NNXUqjB7tZ3iOHx90NbsYPtzvElR91yDtlSLpJlyjXMAHeiz90KmmzE/73+NABz8k78svfWh27w5DhsS3toaYMcMPT+zb178hGY/hiQkwfLgCXNJfav7vqo/o78zp2kJvSKA3aQKPPOKHMA4b5kM0FSxa5Icntm8P06bt2q8hInEVmkB/40PfQu+fvyE9d7tZvtx/rk+ggw/LqVN9eJ5+ug/TIK1b54cnrl/vR+Tst1+w9Yg0AqEI9KIi+PM9PtCz2MDChVBQkGah3pAWetR++/nw3LDBh+m6dfGpbU9VVPgx37Nn+w2S8/ODqUOkkQlFoBcWwsrNPtBb4rtcysv98bQRDfT2DVx1OD/fh+js2T5UgxjOePXVfkf6e+6Bk09O/uuLNFKhCPTSUliJX2OjPSuqHE8bZWV+NmOzZg2/1kkn+XVHpk/34ZpMd93lg/yaa2DEiOS+tkgjF4pRLrm5sGRhFwByWFTleNqIdZZorC691G/8cOutcOCBfpx6oj3/vP8BMmQI/OUviX89EakiFC30ceMgI2svviV7R6Cn3cSQPZ1UFIs//xnOOMOH7PPPx/fa1X38se/i6dPHv3mRkZHY1xORXYQi0KMTQ75tlkMOi9JzYkgiAj0jw68q2KePD9uPP47v9aOWLPFvwrZp44cn1rb8r4gkVCi6XCAS3lNyOPSLL1hQEnQ19VBWBkcfHf/rtmwJ//wn9O/vQ/e//4UuXeJ3/fXr/TDJtWvhnXegc+f4XVtE9kgoWug75OSk2TuhEdu2wcqV8W+hR3Xq5Ltc1q3z4bt+fXyuu22bn8j0ySfwxBNw6KHxua6I1Eu4Aj0314fWmjVBV7JnVq70C2slKtDBh+0TT/jwHTbMh3FDjR7tW/933QWDBjX8eiLSIOEK9Jwc/znoWZJ7Kh6TimJxyilw990+hEePbti17r0X7rwTRo2Cyy+PT30i0iAK9FSQrEAHuOwyuOoqH8b33lu/a0yf7odBnn66HxYpIikhNG+KAgr0WP31r351xpEjYf/996y75JNP/IYahx0Gjz2m4YkiKSRcLfROnfzKg+n2xmiyAz0jw4fxYYf5cP7kk9i+b+lSP1Jm3319t02rVomtU0T2SLgCPTPTD5tLxxZ6RoYfx50srVr5UN53Xx/SS5fu/vwNG3wXy+rVfsRMPIc+ikhchCvQwY90ScdAb98++Zs/dOniw3n1ah/Wta0lv22bH+g/cyZMngy9eye3ThGJSfgCPScnPQM9Wd0t1fXu7UN65kwf2jUNZ7zuOr/W+u23+9a8iKSkcAb64sWpvWFydUEGOviQvv12H9rXXVf1ufvug9tugyuuSM4CXyJSb+EM9E2bYMWKus9NAUVF8PWHy3ns1exgd1oaOdKH9m238d9f3ke3bvAT9xIV/3MFSw4b5ANfRFJaOAMd0mKkS1GR31mpbUUZZWQHv9PS7bez5LBBHP6PKxiy8E6e5BxKyOfwzydT9ES4RriKhFH4Aj26CHoa9KMXFsK28k3sy1rK8F0uge60lJnJSasmU0I+d3IV62nFaTxP2ca902v3J5FGKnyBnkaTi0pLoTtf+a/JrXI8KHMX781pPM/jDOVU/sUSugZek4jEJny/R3foAM2bp0Wg5+ZC/kK/1m8J+VWOByU3FxYu7MowHt/luIiktvC10J2Drl3TItDHjYPembPZRhPmcRAQ/E5L48b5GioLuiYRiU34Ah3SZiz68OFwfp8SFmR+jy1ur5TYaSm6+1Nenv/ZmAo1iUhswtflAj7Q33gj6CpikrumBAbns/3poCvZafhwBbhIOgpnCz03169NEo9NHBJp40aYPx/y8+s+V0SkDjEFunNugXNulnNupnOuuIbnnXPuLufcfOfcp865w+Nfauw+WJoD27aRk7ms1sk6RUXQrZtfPiUZE3pqfL1582D7dgW6iMTFnnS5HGdmtU2/PAU4MPLRH5gQ+Zx0RUXw9GM5PAN0ZRHvL+xKQYF/LtqNEJ3QU17uH0cn9FQ+J9411fR63S6YzUBQoItIXMSry2UI8Ih57wOtnXOd4nTtPVJYCF9s9mPRc/GDp6tP1iks9MfG8CdOYXqN58S7pozytTzMBezHsh2v9+ljJdC0KXzve4l5YRFpVGINdANeds7NcM4V1PB8F6DysJLFkWNVOOcKnHPFzrni5cuX73m1MSgthSWRl96Pb6ocr/x1BhX8nt9zPpNqPCfeNR3Ju1zAo/yK+3ccz1lbAgcd5ENdRKSBYg30gWZ2OL5r5XLn3NHVnnc1fM8uyx2a2UQz62tmfTt06LCHpcYmNxe+ozVbySSbsirHK3/9PebTnC20ZVWN58S7ppzIz7uhTCb6R9M7o0TdLSISNzEFupktjXwuA54F+lU7ZTGQU+lxV6COLXASY9w4aJHVhBW03xHo1SfGjBsHhzebDUAbVtd4Trxr6p7pA70nc8mnhI4t1tJ120IFuojETZ2B7pxr6ZzbO/o1cBJQUu20acAFkdEuRwBrzGxZ3KuNQXRizHdNs8mmrMaJMcOHw5jT/C20ZVXCJ88MHw7nDljEWrcvFWRw6T6Teeh/5/gne/VKzIuKSKMTyyiXjsCzzrno+Y+Z2YvOuREAZnYfMB0YBMwHyoGLElNubIYPBx7K5uD1ZQx5r+ZzDm3iA/3AdqtZsCDxNR3QtBT6Hwz77MMV8ydDl/39E2qhi0ic1BnoZvYVcFgNx++r9LUBl8e3tAbKzoavvqr9+ZLILxmrV/ux4Inez3PRIujTB04+GS6+GB5+GFq0gP33T+zrikijEc6ZouADvays5uc2b4YvvvA732/fDuvWJbYWMx/oOTlw5plsy2gKb7/Nhxt70a17k+A2tBCRUAl3oK9f76fXV/fZZ35ZgIED/eNVq3Y9J55WrvTb4uXkUDS9DS/ZyYBfMjfwXYpEJDTCHegANY13j3a3/OhH/vPq1YmtJbryY04OhYVQtH0oALPxb4gGukuRiIRG+AO9pm6XkhLIzIT+kdUJEt1Cj85YysmhtBSe5Uwe5Jc8zdm7nCIiUl/hXD4X6g70Hj1gv/3840QHerSFnpsb2REoi0t4sMop2hFIRBqq8bbQ8/OhTRv/OBldLs2aQYcO2hFIRBImvIEeXVqgeqBv2ABff+0n9LRt648lo4XetSs0aaIdgUQkYcLb5dKypR/nXT3Q50RmaObn++ebN098C7201A9ZjNCOQCKSCOFtoTtX81j06AiX6AzNtm2T00LPyan7PBGRBghvoMMugV5UBPePKmEje9H9x9392O82bWJqodd7h6Nt22DJEr3rKSIJ12gCPbprUO66EuZyMF+XZlBQAGUVdbfQo9+7cKGf9LlHk4G++caHulroIpJgjSbQo7sU9WI2JfjulvJy+GRR3S306PeewnRu4vod3xvTZKBKk4pERBKpcQS6GaWl0JrVdGXJjhmaAEs21t1Cj076OY/HGcN4mrOpyvGYvlmBLiIJFv5A37oV1q4lN9e3zgFmcciOU7btXXcLPdr9nU0ZmWyjB59VOb5blSYViYgkUvgDHaCszO9S1NSPcIl2uWRlQd+T2vpFvLZurfUy0clA0R2Q8imJfTLQokV+Vcd9923QrYiI1KXRBPrw4XD5sbNZ5/ZmMTk7JvQcdmzds0Wjk4E6ZfhAP3Kf2bFPBooOWXQ1bbsqIhI/4Z1YBLvMFu2xtQT692L7e5XC9bFKs0WjPwBqMHyYwS/8dS4/pgRinRikMegikiSNpoWOGcyateuWb9Hp/3WNRf/uO6io8F+XVN9SdTeqzRIVEUmUcAd65RZ6WZnfaKJ6oEcX6Kprtmh0gtJBB/m1YG80u4EAAArCSURBVNav3+WUypOP2reHzu02w7ffctuUXG1gISIJF+5Ab9YMWrf2YVx9yn9UrC30aKAff7z/HF0TJqL65KOVK2GvVUsAmLUmR7sSiUjChTvQYedY9Nl+yGKDW+jHHec/R68XEZ181JPZ/Iej+IB+/JPTAVhEjnYlEpGEazyBXlIC7drt+sZn69b+c10t9OhWdkccAXvttUs/enT+0GCmcRTvsJJ2LCSPyfyMD+hf5RwRkUQI9ygX8AE+bx5s3uxb59WHD2Zmwj77xN5C79gRevbcJdD9TkR+8lIpOQzihV0uoblFIpJIjaOF/u23voukendLVNu2sfWht20LTZv661QL9Ojko3xKdkxcqky7EolIojWOQF+5EtaurT3Q27SJrYUeHTXTqxcsXVrlh8Dw4XD/hAoOZi6zyaddO9/Do12JRCRZGkeXS1RDW+jRa0WvM3s2HHXUjlOG9ZsPbOHah3px7YX1L1lEpD4aRws9qlevms+JtYVePdCrTzCqbWikiEgShD7QX5npu0kW04VufdrUPBa8Ugu91p2JKgV60Vs5rHN7c+//lFQ9p6TE97EcfHAC70hEpGah7nIpKoLbbsvmRGA2vXbsNATV+rMjLfSiSUbBpY7ycn84er7bVsGwlSshO9tPILrU0d160ZPZVa85ezYccIB/B1REJMlC3UIvLISFm3yruvIuRbtM8GnbFrZu5ebflFNeDqO4g6u5bcf5txeu8OdlZ++YQFRCPocwC7Cd1ywpUXeLiAQm1IFeWgoracfNFPIPLqpyvIrIbNENi1bRnE38gRu4gnt2PL15cWRSUXb2ju/9Dz+iPSvpx38B+HbhJvjii9r76UVEEizUge4n8jh+y83MrjQ2fJcJPpH1XHp2Ws1PeJF9WEd3vqYlfgGu/OzIpKLs7B3fO5UhbKYZQ5kMwLGdPvObQauFLiIBCXWgRyf7VFbjBJ9IoI++aBXDMybvONyTOWRlwWU/3Rno0WuuZV+mM4hzeZKWLbZz/Rka4SIiwQp1oEd3GsrLq2OCT6TL5cSDFnFm5j/5z14/BuCYdiVMnAhHfT8S6B06VLnmk/yMLizl2dFvM7D1bL+MwPe/n8Q7FBHZKdSjXMCHd50zNKNL6D76KJmby/nRq2Ph1Le55cLZfmeiwjLIyNgR/DuuueF0yM7ixJWTYfFi6NHDL9krIhKA0Ad6TKJL6L7yCnTuDMccU3UBrui0/ybVfqFp2RJOOw2mTIEWLWDAgOTWLSJSScxdLs65DOfcx86552t47ljn3Brn3MzIxw3xLTOxiqa2YmvkZ9vf151D0eSMqgtwVZ4lWs2bnYb6pXVLS7n1xV7axEJEArMnfeijgLm7ef4/ZtY78vGHBtaVNNGJQqvxrfSJ64ZSUAAfbcn3C3CtWlVroBcVwZkTT2EtewPw9pp87UwkIoGJKdCdc12BU4EHEltO8kUnCq2iLQvI4wP6U14Od70WGU8+e3atgV5YCKs37sVznAH4yUbamUhEghJrH/odwHUQaYrWbIBz7hNgKfBrM5td/QTnXAFQAJCbIrs9RCcK/YmxrGFfwG+A8XpZpRUVly+vMdArf+9SOvMlB1Q5LiKSTHW20J1zpwFlZjZjN6d9BOSZ2WHA3cBzNZ1kZhPNrK+Z9e0QXVs8YNGfK49wIVMjLW0Al5sDe+8NH34I69bVGOjR753HwYxlPBb540yRn1Ui0sjE0uUyEBjsnFsATAaOd85NqnyCma01s/WRr6cDTZ1z7eNdbCLUOvnoj86/Mfr66/5gDYEe88QlEZEkqDPQzWysmXU1s27AUOA1Mzu/8jnOuf2c85t1Ouf6Ra67MgH1xt1uJx/l58PXX/sTa/iNIuaJSyIiSVDvcejOuREAZnYf8FPgf5xzFcBGYKiZWXxKTLxaJx9VXmirlmGLMU1cEhFJgj0KdDN7A3gj8vV9lY7fA5WWJwyLyuuy1BLoIiKpItRruTTU05/tDPSex2ZrfLmIpDQFei2KiuCCX2eznPaU04K5i1pq0pCIpDQFei0KC6F8o6OEfJbTAXCaNCQiKU2Lc9Wi8qSh/fhml+MiIqlGgV6L3Fy/SfQrnLTLcRGRVKQul1po0pCIpBsFei00aUhE0o26XHZDk4ZEJJ2ohS4iEhIKdBGRkFCgi4iEhAJdRCQkFOgiIiHhglrl1jm3HFi4B9/SHliRoHJSWWO878Z4z9A477sx3jM07L7zzKzGLd8CC/Q95ZwrNrO+QdeRbI3xvhvjPUPjvO/GeM+QuPtWl4uISEgo0EVEQiKdAn1i0AUEpDHed2O8Z2ic990Y7xkSdN9p04cuIiK7l04tdBER2Q0FuohISKRFoDvnfuKc+8w5N985NyboehLBOZfjnHvdOTfXOTfbOTcqcrytc+4V59wXkc9tgq413pxzGc65j51zz0ceN4Z7bu2cm+Kcmxf5Ox/QSO776si/7xLn3OPOub3Cdt/Oub8758qccyWVjtV6j865sZFs+8w5d3JDXjvlA905lwHcC5wC9ATOc871DLaqhKgARpvZwcARwOWR+xwDvGpmBwKvRh6HzShgbqXHjeGe7wReNLODgMPw9x/q+3bOdQFGAn3NLB/IAIYSvvt+CPhJtWM13mPk//hQoFfke/4vknn1kvKBDvQD5pvZV2a2BZgMDAm4prgzs2Vm9lHk63X4/+Bd8Pf6cOS0h4EzgqkwMZxzXYFTgQcqHQ77Pe8DHA08CGBmW8zsO0J+3xGZQAvnXCaQBSwlZPdtZm8Bq6odru0ehwCTzWyzmX0NzMdnXr2kQ6B3ARZVerw4ciy0nHPdgD7AB0BHM1sGPvSB7OAqS4g7gOuA7ZWOhf2euwPLgX9EupoecM61JOT3bWZLgL8CpcAyYI2ZvUzI7zuitnuMa76lQ6C7Go6Fdqylc64V8DRwlZmtDbqeRHLOnQaUmdmMoGtJskzgcGCCmfUBNpD+3Qx1ivQbDwH2BzoDLZ1z5wdbVeDimm/pEOiLgZxKj7vif00LHedcU3yYF5nZM5HD3zrnOkWe7wSUBVVfAgwEBjvnFuC70o53zk0i3PcM/t/0YjP7IPJ4Cj7gw37fPwa+NrPlZrYVeAY4kvDfN9R+j3HNt3QI9A+BA51z+zvnmuHfQJgWcE1x55xz+D7VuWZ2W6WnpgEXRr6+EJia7NoSxczGmllXM+uG/3t9zczOJ8T3DGBm3wCLnHM9IodOAOYQ8vvGd7Uc4ZzLivx7PwH/XlHY7xtqv8dpwFDnXHPn3P7AgcB/6/0qZpbyH8Ag4HPgS6Aw6HoSdI9H4X/V+hSYGfkYBLTDvyv+ReRz26BrTdD9Hws8H/k69PcM9AaKI3/fzwFtGsl93wjMA0qAR4HmYbtv4HH8ewRb8S3wi3d3j0BhJNs+A05pyGtr6r+ISEikQ5eLiIjEQIEuIhISCnQRkZBQoIuIhIQCXUQkJBToIpU457pVXiVPJJ0o0EVEQkKBLlIL51z3yOJZPwy6FpFYKNBFahCZlv80cJGZfRh0PSKxyAy6AJEU1AG/1sbZZjY76GJEYqUWusiu1uDXqB4YdCEie0ItdJFdbcHvKPOSc269mT0WdEEisVCgi9TAzDZENuB4xTm3wczCuKSrhIxWWxQRCQn1oYuIhIQCXUQkJBToIiIhoUAXEQkJBbqISEgo0EVEQkKBLiISEv8PRT0zOPMCjCAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "k_list = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,30,40,50,60,70,80,90,100]\n",
    "p_list = []\n",
    "for k in k_list:\n",
    "    p = cross_validation(norm_mat, labels, k)\n",
    "    p_list.append(p)\n",
    "# k_list = np.array(k_list)\n",
    "# p_list = np.array(p_list)\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "ax.plot(k_list, p_list, c='red')\n",
    "ax.scatter(k_list, p_list, c='blue')\n",
    "plt.xlabel('k')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
